ENTROPY FOR SYMBOLIC DYNAMICS WITH 
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Abstract. We consider shift spaces in which elements of the alphabet 
may overlap nontransitively. We define a notion of entropy for such 
spaces, give several techniques for computing lower bounds for it, and 
show that it is equal to a limit of entropies of (standard) full shifts. When 
a shift space with overlaps arises as a model for a discrete dynamical 
system with a finite set of overlapping neighborhoods, the entropy gives 
a lower bound for the topological entropy of the dynamical system. 



1. Introduction 

There is a long history of using symbolic dynamics to model more compli- 
cated dynamical systems. Suppose, for example, that we want to understand 
the dynamics of a continuous dynamical system / : X — > X on a compact 
metric space X. If we have compact sets N\, . . . ,N n C X and an /-orbit 
(x, f(x), / 2 (x), . . .) that remains in (J^ JVj, then there is a symbol sequence 
(io,ii,...) G {l,...,n} N called an itinerary such that / J (x) € for all 
j > (the itinerary may not be unique). We can then use the properties of 
the set of itineraries (a shift space) to tell us about the dynamics of /. 

One example of this approach is a Markov partition, in which the sets 
N%,..., N n C X cover the entire space, overlap only on their boundaries, 
and map across each other in topologically simple ways ([61 §9.6]). In this 
case the resulting shift space of itineraries is a subshift of finite type. Another 
example is when the iVj's are pairwise disjoint Conley index pairs , resulting 
in a cocyclic subshift ( [U EJ [3] ) . 

While these approaches have been very fruitful, there are limitations with 
each of them. Markov partitions can be very difficult to construct in practice, 
and the Conley index pairs must be disjoint, making it impossible to cover 
a connected space. 

In this article we allow the Ni to have nontrivial intersections. The com- 
plication that arises if we allow the iV,'s to intersect is that an orbit may 
have multiple itineraries. For example, a fixed point in iVi n N2 could have 
itineraries (1, 1,1,.. .), or (1, 2, 1,2,.. .), or any sequence of l's and 2's. Our 
task is to extract useful dynamical information about / from the shift space 
while being careful about the nonuniqueness of itineraries. 

We begin with the definition of our object of study — a shift space with 
overlaps. 
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Definition 1. A shift space with overlaps is a pair (X,X) in which X is a 
one-sided shift space on the alphabet {1, . . . ,n} and X is a simple graph 
with vertex set { 1 , . . . , n} . We call X an intersection graph. Two words 
(ao, ai, ■ ■ ■)> (&0i &i> ■ • •) £ £ (finite or infinite) are indistinguishable iff there 
is an X-edge between the vertices a, and 6, for all i. 

Shift spaces with overlaps are interesting objects to study on their own, 
but most often they are a model for an existing dynamical system. Thus we 
have the following definition. 

Definition 2. A compact metric space X, a continuous map / : X — )■ X, 
and a collection of nonempty compact sets N\, . . . , N n C X is called a 
dynamical realization of (S,X) provided: 

(1) If Ni D iVj 7^ and i ^ j, then there is an edge from i to j in X. 

(2) If (ao, oi, • • ■ ) is a (finite or infinite) word in £, then there is a point 
x S X such that f l {x) S Int(iV ai ) for all i. 

In practice, we use topological methods to verify that (2) holds. The 
methods may be elementary, such as boxes that stretch across other boxes, 
as in the case of Markov partitions or their topological generalizations shown 
in Figure [lj Or they may be more sophisticated; for example, in [5] we use 
techniques from Conley index theory to verify the property. 




Figure 1. 



For much of this article we assume that £ is a subshift of finite type. When 
£ is the vertex shift associated to a transition graph T we superimpose the 
graphs T and X with the edges of T being solid arrows and the edges of X 
dashed segments (see Figure [2j for example) ; we will refer to this merged 
graph as TX. 

We are using (£,X) as an abstract model of our dynamical system / : 
X —7- X. As such, two indistinguishable elements in S could be itineraries 
of the same point in X. Thus in a sense we must treat two indistinguishable 
elements in £ as the same point. However, what makes this scenario inter- 
esting is that indistinguishability is not a transitive property, and is thus 
not an equivalence relation. In general, (£,X) is not a topological space. (It 
is a tolerance space — see, for example, |7J.) 

Our goal is to extract information about the dynamical complexity of / 
from (E,X), and most often in this article, from the graph XX. We define a 
notion of topological entropy for shifts with overlap, %(£), in Section[2]and 
show that it is a lower bound for the entropy of a dynamical realization /. 
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We estimate hx(E) using only disjoint sets in Section [3j discuss decompo- 
sitions of the shift space in Section Hi and estimate fox(S) using connected 
components in Section [5] In Section[6]we discuss higher shifts, and in Sec- 
tion [7] we prove that /ix(£) can be computed as a limit of the entropies of 
ordinary (nonoverlapping) embedded shifts. 

One of the motivations for undertaking this study is its potential appli- 
cation to computational dynamical systems. There is an active research 
program of using the Conley index and computational topology to give rig- 
orous entropy bounds for dynamical systems. In particular, one could use 
rigorous computation to obtain an index system (as described in [5]) and 
from that a shift space with overlaps. So it is important to find techniques 
for computing the entropy, or bounds on the entropy, of shift spaces with 
overlaps. 

2. Topological entropy of shift spaces with overlap 

We are primarily interested in using (£,X) to help us obtain a lower bound 
for the entropy of /. Despite the fact that (S,X) is not a topological space, 
we can make a reasonable definition of its topological entropy, and we prove 
that it is a lower bound for the entropy of / (Theorem [3]). 

If we ignore the intersections, then E is a shift space and we can compute 
/i(E) by computing the growth rate of the number of words of length n as n 
goes to infinity (|H §4.1]). Let B n (Y>) be the set of words of length n; then 
the topological entropy of £ is 

fc(E) = lim -log|0 n (E)|. 

n— >oo n 

Recall that if the shift space E is a subshift of finite type with transition 
graph T , then there is a particularly easy way to compute the entropy. The 
graph T has an associated transition matrix, Af, in which the (i, j')-entry is 
1 if there is an edge from the ith vertex to the jth vertex and is otherwise. 
Then 

fc(E) = log(A), 

where A is the Perron eigenvalue of Aq — the unique largest real eigenvalue 
of A T . 

In our setting, we want to count the number of words of length n that 
are mutually distinguishable, which in general is less than |i3 n (E)|. (For 
example, if vertices 1 and 2 overlap, then we would not be able to distinguish 
the words (1, 3) and (2, 3).) Let B be a set of words in E of length n. We 
say that B is n-separated if no two words in B are indistinguishable. Let 
£>n 6p (E,Z) be an n-separated set of maximum cardinality. We define /ix(E), 
the topological entropy of a shift space with overlaps, to be 

%(£) =limsup-log|£C P (£,Z)|. 

n— >oo Tl 

The following theorem provides the motivation for studying the entropy 
of shifts with overlaps. 
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Theorem 3. If f is a dynamical realization o/(E,Z) ; then h(f) > /ii(E). 

Proof. Let e = min d(N, Nj), where the minimum is over all disjoint pairs iVj 
and Nj. Then an n-separated word in E corresponds to an (n, e)-separated 

orbit for /. Since h(f) = limlimsup — log r(n,e), where r(n,e) is the max- 

imum cardinality of a set of (n, e)-separated orbits ([6., §8.1]), the result 
follows. □ 

The rest of the paper is devoted to computing or finding lower bounds for 

Ms). 

3. Independent subsets of the intersection graph 

In this section we take advantage of the fact that if all words are distin- 
guishable, then we can treat our shift space with overlaps as an ordinary 
shift space. 

Proposition 4. Let (E,X) be a shift space with overlaps in which the edge 
set of I is empty. Then /ij(E) = /i(E). 

Corollary 5. Let (E,Z) be a shift space with overlaps. If E' C E is a 
shift space in which all elements are pair-wise distinguishable, then /ix(E) > 
hi(E') = h(Z'). 

Let V(Q) and E(Q) denote the vertex and edge sets of a graph Q, respec- 
tively. Given a (nondirected) graph Q, a subset V <ZV(G) \s independent if 
there are no edges between any pair of vertices in V. The maximum cardi- 
nality of an independent subset of Q is the independence number of Q and 
is denoted md(G). 

For any V C V(l), let Ey = {(a , a 1} . • .) € E : a» 6 V}. Notice that if 
E is a shift space, then Ey is also a shift space. A vertex-induced subgraph 
of a graph Q is a subgraph Q' with the property that any C?-edge whose 
endpoints are in V(Q') is in E{Q'). If V C V(T), let Xy denote the vertex- 
induced subgraph of X with vertex set V. 

Corollaries |6] and [7] give us a tactic for finding a lower bound for /ix(E). 
Remove enough vertices of X so that we have an independent set of vertices; 
then the corresponding shift space with overlaps has no overlaps — it is just 
a shift space. The entropy of this shift space is a lower bound for /ij(E). 

Corollary 6. IfVc V(I) is an independent set, then ftx(E) > /ii v (Ey) = 

In the following corollaries we assume that E is a shift of finite type with 
transition graph T and intersection graph X. 

Corollary 7. Let V C V(I) be an independent set and let T'X' be the 
vertex-induced subgraph of TX with vertex set V. Then /ix(E) > log(A) 
where A is the Perron eigenvalue of A-yt . 

Corollary 8. Suppose T is a complete digraph. Then /ix(E) > log(ind(X)). 
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Proof. Let V be a maximum cardinality independent subset of I. The 
vertex-induced subgraph of T with vertex set V is a complete digraph with 
ind(X) vertices. By Corollary [7J 7ix(E) > log(ind(X)). □ 

4. DECOMPOSITION INTO IRREDUCIBLE AND PRIMITIVE COMPONENTS 

We begin this section with some basic definitions from graph theory and 
linear algebra. (See [U ch. 4] or any non-introductory linear algebra text for 
more details.) 

Let Q be a directed graph. A vertex path (ao, ai, • • • , ai-i) of length I is 
a sequence of vertices in Q such that there is an edge from vertex a\ to aj+i 
for each i, < i < Z — 2. In what follows, we assume that Q has n vertices 
and no parallel edges and that every vertex has at least one edge leaving it 
and one edge entering it. Then the adjacency matrix for Q, Ag, is an n x n 
(0,l)-matrix with no row or column containing all zeros. Notice that for 
k > 1 the matrix Ag need not be a (0, l)-matrix. However, if the (i, j)-entry 
of Ag is nonzero, then there exists a vertex path of length k + 1 from vertex 
i to vertex j (actually, the value of the gives the number of such vertex 
paths). 

A graph is irreducible provided there is a vertex path between any two 
vertices, and a matrix is irreducible if it is the adjacency matrix for an 
irreducible graph. It is always possible to decompose a graph into its ir- 
reducible components; the corresponding matrix is (after rearranging the 
order of the vertices, or equivalently, conjugating by a permutation matrix) 
in block-triangular form with each of the blocks on the diagonal the adja- 
cency matrix for one of the irreducible components of the graph. 

An irreducible graph Q may exhibit cyclic behavior. In particular for each 
vertex i, Per(i) = gcdjra : there is a vertex path of length n+1 from % to itself} 
exists and is the same value for all i. This common value is called the period 
of Q, denoted Per(C7). Define the period of an irreducible (0, l)-matrix A, 
Per (A), to be the period of the associated graph. If the period of the graph 
or the matrix is 1, then we call it primitive. If A is an irreducible matrix 
with p = Per (A), then (after reordering vertices) A p is a block triangular 
matrix with primitive matrices along the diagonal. Finally, if A is a prim- 
itive matrix, then A k is eventually positive. The index of primitivity of A, 
J (A), is the least integer k such that A k > 0. 

Now suppose our shift space with overlaps has an associated graph TI; 
that is, we consider (E,X) where E is a subshift of finite type with transition 
graph T . We make the standing assumption that every vertex of T has at 
least one edge leaving it and one edge entering it; if not, we can remove 
the "stranded" vertices without affecting the dynamics. Recall that the 
graph-theoretical notions of irreducibility and primitivity have dynamical 
interpretations for the shift: T is irreducible iff E is topologically transitive, 
and T is primitive iff E is topologically mixing. 
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Theorem 9. Suppose T is primitive with index of primitivity 7. Then 
log(md(X)) 

7 

Proof. Let {b\, . . . , b- md ( X )} be an independent set of vertices. Since A 7 > 0, 
for any pair b{, bj, there is a word (bi, 02, 03, ... , a 7 , fej) in S. More generally, 
for any n and any b^ , . . . , bi n , there is a word 

{bii ) a 2) • • • j O7; fr« 2 ' a 7+2j • • • ! a 27) • • • > bi„) a (n-l)7+2> ■ • • ) Orry) 

in S. The collection of all ind(X) n such words is ^-separated, so we have 
that \Bn^(E, J) I > ind(X)", and thus 

= limsup 1 log|^ e P(S,X)| > limsup — log|Z»£,Z)| > lo s( md ( X )) , 

□ 

Corollary 10. IfTis primitive with n vertices, then > J* ' — - -j . 

— 2n + 2 

Proo/. This follows from the fact that 7(T) < n 2 - 2n + 2 (|U Thm. 4.14]). 

□ 

In general 7" is not primitive. In this case we break down T into its 
irreducible components. Each irreducible component has some period p, 
and can thus be decomposed into p primitive components. We then apply 
Theorem [9] to obtain the following corollary. 

Corollary 11. Suppose T' C T is a primitive component with index of 
primitivity 7 that resides in an irreducible component of period p and that 
I' is the associated intersection graph. Then 

> MMgj) 
VI 

Corollary 12. If there are two vertices ofT that are in the same primitive 
component and are not joined by anZ-edge, then has positive entropy. 

Example 13. Let U C M? be an open set and / : U — > M? be a continuous 
function that maps the sets N\, . . . , Nu C U as shown in Figure [2j The 
associated graph TT is shown on the right. 

First we remove vertices 6 and 9 since they lack an outgoing and an 
incoming edge in T, respectively. (Note: if we also removed two of the 
vertices 1, 2, and 4 to obtain an independent set, then we could apply 
Corollary [7J but the corresponding shift space would have zero entropy.) 
The resulting transition graph has two irreducible components, with vertex 
sets {8} and {1, 2, 3, 4, 5, 7, 9, 10, 11}. The second of these has period 2 and 
it decomposes into the primitive components {3,4,7,11} and {1,2,5,10}. 
It is straightforward to show that both of these have index of primitivity 
4 and that the first of these is an independent set. Thus we conclude that 
h(f) > %(£) > 2^4 l°g(4) ~ 0.173. We will return to this example later. 
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Figure 2. 



Example 14. Let S 1 be the circle, viewed as M/Z and let / : S 1 — > S 1 be a 
map that is C°-close to the doubling map (x h-» 2x). Let iVi = [—0.1,0.35], 
N 2 = [0.15, 0.6], iV 3 = [0.4, 0.85], and iV 4 = [0.65, 1.1]. The transition matrix 
and the graph TX are shown in Figure [3j 




Since Aj- consists of all l's, Aj- is irreducible with Per (Aq-) = 1 and 
primitive with 7(^7-) = 2. The maximal independent sets in X are {1,3} 
and {2,4}, so ind(X) = 2. Thus h(f) > h x (E) > |log(2), which is half the 
entropy of the doubling map. We will return to this example. 
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5. SOFIC SHIFTS AND X-COMPONENT SHIFTS 

Vertices that are adjacent in X are indistinguishable. This is a problem for 
our symbolic dynamics, since different symbols may not correspond to differ- 
ent points. One way of working around this problem is to give neighboring 
vertices the same label. In other words, give each vertex in a connected 
component of X the same label. 

Suppose X has connected components Ai, . . . ,A r . Then create the shift 
space Sj on the alphabet {^i, . . . , A r } as follows. The element (Ai Q , A{ x , . . .) G 
Tjx if and only if there is an element (ao, a%, . . .) € S such that G Ai k for 
all k. We call Sj the Z-component shift space for (S,X). 

Theorem 15. Let Sj be the Z-component shift space associated to (S,X). 
Then h x (E) > h{T, x ). 

Proof. Let £> m (Xx) be the set of words of length m in £j. For each element 
(A io , . .. , Aj^J G B m (Ez), pick one word (a , . . . , a m _i) in £ such that 
a k G Ai k for k = 0, . . . , m — 1. The collection of these words in £ are 
m-separated. However, it may not be a maximal m-separated set. Thus 
|i3^ p (S,X)| > |0 m (S x )|, and hence > K^x). □ 

A set of vertices in a nondirected graph form a clique if every pair of 
vertices in the set are joined by an edge. 

Proposition 16. // the vertices in each connected component of Z form a 
clique, then /ix(^) = M^z). 



Proof. By Theorem 15 all we must prove is that < Let 

£>m P (£, X) be a maximal m-separated set. Consider the function ip : £>m P (S, X) — >• 
#m(£x) given by i/j(a ,ai, . . . ,a m _i) = (A , Ai,- • where a fc G A ifc 

for all k. We will prove that this function is injective. Suppose ip(ao, a 1} . . . , a m _i) 
^(6o,&i, • ■ -j&m-l) for some (a ,ai, . . . ,a m _i), (&o,&l, • • • ,&m-i) G Bm P (S,X). 
Then ak,bk G Aj fc for all k. But the subgraphs of X with vertex sets 
j4j fe are cliques, so and are the same or are indistinguishable. Thus 
(ao, ai, . . . , a m _i) and (&o, . . . , 6 m _i) are indistinguishable. Since £>m P (£,X) 
is m-separated, (ao, ai, . . . , a m _i) = (6o 5 bi, ■ ■ ■ , b m ~i), and V is injective. 
Because ip is an injective function between finite sets, we conclude that 
|23^ P (£,X)| < |£ m (£z)|. Hence %(E) < n(E x ). □ 

An important special case occurs when S is a subshift of finite type. 
Then Sj is a sofic shift (see [U Ch. 3] for more about sofic shifts and the 
terms in this paragraph). If (S,X) is represented by the graph TX, then 
Sj is the sofic shift represented by the directed graph obtained from T by 
giving all vertices in a connected component of X the same label. If this 
graph is right-resolving (that is, if every edge from a given vertex points 
to a different label), then the entropy of the sofic shift is simply the log of 
the Perron eigenvalue of the adjacency matrix for T ' ■ If the graph is not 
right-resolving then there is a mechanical procedure for transforming it into 
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a right-resolving graph representing the same sofic shift. In particular, the 
entropy is easy to compute. 

Example 17. Consider the graph TX shown on the left in Figure [4} If 
we give vertices 1 and 3 the same label we obtain the "golden mean" shift 
shown on the right (a shift that is well-known to be sofic but not of finite 
type). By Proposition 16 the entropies of the two are equal, and since the 



graph for the sofic shift is right-resolving, we can easily compute the entropy 
from the adjacency matrix: = h(T>z) = log(|(l + 





Figure 4. 



Example 18. Consider the sofic shift associated to the shift space with 



overlaps in Example 13 (with vertices 6, 8, and 9 removed). That is, we give 
vertices 1, 2, and 4 the same label (the graph on the left in Figure [5]). The 
graph is not right-resolving, but after being put in right-resolving form (the 
graph on the right) we find that %(£) > log (g(108 + 12 v / 69) 1/3 + 2(108 + 
12v / 69)" 1/3 )) » .281, which is a larger lower bound than the one obtained 
in Example |13[ 




Figure 5. 



Example 19. On the other hand, the sofic shift corresponding to the dou- 
bling map of Example 14 has only one symbol, so we obtain a lower bound 
of zero for the entropy. 
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In some circumstances we could obtain a larger lower bound for the en- 
tropy by using a combination of removing vertices (as described in Section 
[3]) and relabeling (as described in this section) than we could using either of 
these techniques alone. We could remove a few vertices to increase the num- 
ber of connected components of X, then relabel to obtain an X-component 
shift with high entropy. 

6. Higher shifts 

In this section we examine higher shifts. This is a well-known way of 
altering the transition graph for a subshift of finite type to obtain a new 
graph which generates a topologically conjugate subshift of finite type. The 
higher vertex shift allows us to subdivide the set of vertices, thus presumably 
giving us a greater number of disjoint vertices. We show how to implement 
this notion for shift spaces with overlap generated from a graph TX. We 
refer the reader to [U Sect. 2.3] for details on higher shifts for edge-labeled 
graphs; the constructions can be modified easily for vertex-labeled graphs, 
which is what we need. 

For m > 1 we define the mth higher vertex graph 7r m i to have vertex set 
equal to the collection of all vertex paths of length m in T, with an edge 
from a vertex (io, ■ • • , im-i) to a vertex (i\, . . . , i m -i, im) provided there is 
an edge in T from i m ~i to i m . Note that the first higher vertex shift 7m is 
simply T. 

The intersection graph X induces an intersection graph X[ m i for the vertices 
of T\ m y There is an (undirected) edge between the vertices (io, . . . ,i m -i) 
and (jo, • • • , jm-i) if the words are indistinguishable; that is, if there is an 
edge in X between i^ and jk for all k. 

Theorem 20. If T[ m ]I[ m ] is the graph associated to the mth higher shift of 
(£ T ,Xj 3 then h X[m] (^r [m] ) = %( S r)- 

Proof. Let m > 1 be fixed. For each n, there exists a natural bijective 
function tp n : B n+m -i(T, T ) -> B n (T, T[m] ). Specifically, ip n (bo, b n+m - 2 ) = 
(wo, . . . ,w n —i), where Wi — (bi, . . . , ^m+t— 1 

). Moreover, ip n (bo, • ■ • , 
and Vn(cO) • • • j Cn-l) are distinguishable if and only if (bo, ■ ■ ■ , b n -i) and 
(cq, . . . , Cn-i) are distinguishable. Thus, ^l^^r ,Z)\ = |i^ ep (£^ m] ,X [m] )|, 
and hence 

hx lm] (Etm ) = lim sup \ log | B%> , X [m] ) | 
= lim sup - log \B n e l m ^ (S r , X) | 

n— >oo To 

= %(Sr). 

□ 

Proposition 21. ind(Xj m ]) > ind(X). 
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Proof. (Recall that we are assuming that each vertex has at least one edge 
leaving it.) If {61, . . . , 6md(x)} is an independent set of vertices of X, and Wi = 
(bi, a^, . . . , ai m _ 1 ) is, for each b{, any word of length m, then {w\, . . . , w ind ^} 
is an independent set of vertices of Xr m i . 

□ 

Proposition 22. Assume that T has at least two vertices. IfT is primitive, 
then so is 7r m i, and 7(7r m ]) = 7(7") — 1 + m. (IfT has only one vertex, then 
7T m i is isomorphic to T .) 

Proof. A graph is primitive with exponent 7 if and only if from any vertex 
to any other there is a vertex path of length 7 + 1, and 7 is the smallest 
number with this property. Observe that (i m -i, ki, . . . , k p ,jo) is a vertex 
path in T (of length p + 2, with p > 0) if and only if 

! im—l)i {il j • • • 1 ^m— 1 j • • • > (kpi jOi ■ • • ) jm—2 

is a vertex path in 7J m ] (of length p + m + 1) for any vertices (io, . . . , i m -i) 
and (jo, . . . , jm-i). Between any two vertices of T there is a vertex path of 
length p + 2 = "y(T), so between any two vertices of 7[ m ] there is a vertex 
path of length p + m + 1 = 7(T) — 1 + m. Thus 7(7[ m j) < 7(7") — 1 + m. 

To prove the opposite inequality, first assume that 7(7") > 1. Then there 
exist two vertices of T such that there is no vertex path from the first to the 
second of length 7(7"), and thus two vertices of T[ m ] with no vertex path of 
length 7(7") — 1 + m between them, so 7(7[ m j) > 7(7") — 1 + m. 

Finally, if 7(7") = 1, then any vertex of T can follow any other vertex. 
For i ^ j, it is clear that the shortest vertex path in T[ m ] from (i, . . . , i) to 
(j, . . . , j) has length m + 1, so 7(77 m i) > m = 7(7") — 1 + m. 

□ 



Example 23. We return to Example 14 The vertices of T\2\ are (1, 1 



(1,2), (2,3), (2,4), (3,1), (3,2), (4,3), and (4,4)— one for each edge (or 
vertex path of length 2) in T ■ There are nine edges in Xm ■ They can be 
seen in the graph TpyEw in Figure [6| 





Figure 6. 
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Since 7(7") = 2, Proposition 22 says that 7(7[2]) = 3, as we can check by 
observing that AL, has zero entries while AjL does not (where Aj 2 ] is the 
adjacency matrix for 7^2] ) • 

Notice that the set of vertices {(1, 1), (2, 3), (3, 1), (4, 3)} is a maximum 
cardinality independent set in Zp], so ind(Z[ 2 j) = 4. Theorems [3j [9j and 
tells us that 

log 4 2 log 2 
h(f) > %(S T ) = % [2] (^ 2] ) > -|- = 



20 



which is greater than ^log2, the lower bound we obtained in Example 14 
using 7". In this particular example we see that 7(7j m i) = m + 1 ancT 
ind(Z [m] ) = 2 m . So 

/j( n > log 2m = ml °g 2 
~~ m + 1 m + 1 

for all m. This implies that h(f) > log 2, the entropy of the doubling map. 

7. Entropy as a limit 

In Example [23] we see that we obtain a very good entropy bound by 
looking at the sequence of higher shifts. This is true for more than this one 
example. In this section we show that when the shift space with overlaps is 
given by a graph TZ, the sequence of higher shifts is closely related to the 
entropy. In particular, if 7" is primitive, the entropy can be expressed as a 
limit. 

As in Section [6j let T[ m ]I[ m ] denote the graph for the mth higher shift. In 
this case £> m (E) = V(T\ m ]I\ m ]). Moreover, an m-separated set B in S cor- 
responds to an independent set in Z m i, and hence |£>m P (£,Z)| = ind(Z[ m j). 
Thus /ix(S) is the growth rate of the independence number of Z[ m ] as m 
goes to infinity. 

Theorem 24. // (S,Z) is the shift space with overlaps associated to the 
graph TI, then 

^(S) = limsup l0g(ind(J H)) , 

IfTis primitive, then 

u / v*\ v lo g( m d(Z [rn] )) log(ind(Z [m] )) 
%(£) = hm ln - \ = hm • 

m->oo 7(/[ m ]J m-!>oo m 

(In particular, both of these limits exist.) 

Proof. The limit superior equality follows from the definition of entropy and 
the discussion preceding this theorem. Suppose T is primitive with index of 
primitivity 7(7"). Then T[ m ] is also primitive and 7(7J m ]) = 7(7") +m — 1. 
So it follows that 

, r log(ind(Z [m] )) log(ind(Z [m] )) 
%(2j) = limsup = nmsup — — 
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Clearly 

log(ind(Xr m] )) log(ind(Jr m i)) log(ind(J [m i)) 
lim sup > Iim mi = lim ml — — — - . 

m->oo m m-s-oo m m->oo j{T[ m ]) 

So to complete the proof it is enough to show that 
iiminf l g(md(X [m] ))^ % 

m— >oo in 

Since we have ind(X[ m ]) m-separated words and T is primitive, we can 
make from them (ind(Xr m ]))L^/W[ m ])J M-separated words by concatenating, 
as in the proof of Theorem [9| Thus 

ind(X [M] )>ind(X w )L A W™])J, 

and therefore 

v . . log(ind(Jr M i)) log(ind(Xr m i)) 

hmmf — - — > — — ^ — . 

M-+00 M 7(7f m ]) 

Since this holds for all m, we have that 

log(ind(X [M] )) log(ind(X w )) 
lim mi — > limsup — — r = %(2j). 

M^oo M m^oo 7(/[m]) 



□ 



, / V x . J - log(ind(X [m] )) - l 
fcr(S) < sup \ — — \. 



Corollary 25. 7/7" is primitive, then 

u rlog(ind(X H ))-, 
fcr(S) = sup <^ -— k 

Proof. It is obvious that 

log(ind(X H )) 

m>0 l- l(T[ m ]) 

We must prove the reverse inequality. To do this, observe that since we 
have ind(X[ m j) m-separated words and T is primitive, we can make them 
into ind(Xr m i) fc words of length k^(T\ m ]) (for any k) by concatenating. Thus 
ind(X [fc7(7fm])] ) > ind(X [m] ) fc , so 

log(ind(X [fc7(7[m])] ) > log((ind(X H )) fc ) = log(ind(X H )) 
kl(T [m] ) ~ kj(T [m] ) 7(7[m]) 
By taking the limit of both sides as k — > oo, we obtain 

log(M(2, ,)) , 

7(/[m]) 

since this holds for any m, we are done. □ 
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Example 26. Recall from Example 23 that 7[ m ] is a primitive graph with 



index of primitivity m + 1 and independence number 2 m . By Corollary |25[ 

w f w, m flog(?nd(Z[m]))\ f log2 m -( j- mlog2 ) 

/*(/) > M S ) = sup <^ — \ = sup i — \ = sup \ — \ = ln2. 

Theorem 24 gives us the same bound, 



yrn 



h(f)>h x (X)= lim M^H» = lim ^ = ln2 , 

m— >-oo 771 m— »oo 777, 

and happens to be the limit of a constant sequence. 

r log(ind(Xr m i)) -> 

The following example shows that the sequence < r > may not 

1 j{l[m\) J 

be nondecreasing. 

Example 27. Consider the graph 7~X shown in Figure [7| 




Figure 7. 

It is easy to check by hand that 

logtindgpi)) = i i log(md(X| 21 )) 

7(7[i]) 2 3 1 ' 7 (7; 2] ) 

Thus the sequence is not nondecreasing. 

We used a computer to find the first several terms of the sequence; based 

on this, the pattern appears to be \ ln(2 m ) > where m = n/2 for 

L n + 1 J n=i 

n even and m = (n + l)/2 for n odd. Simplified, the nth term becomes 
77- In 2 

when n is even and \ ln(2) when n is odd. The two values given by 



2(n + l) 2 
the theorem (the supremum and the limit superior) are equal and the limit 
superior is equal to the limit. In particular, this common value, \ ln(2) was 
the first term in the sequence! 
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